1 Problem: Timelapse images are packed with information

Between fixed cameras, satellite images, fly-overs and a variety of other sources, there are ever more archives of images with high (~minutes) temporal information about landscapes, construction projects, mining, and a number of other fields. These images can easily exceed hundreds of terabytes together and contain detailed information of the movement of people, deforestation, water-levels, and …. This information is, however, locked up inside the images and expensive to extract manually.

Expensive!

2 Solution: Ad-hoc Querying of Archives.

By making the archives of images not only acessible but easy to process using SQL queries, the data can be immediately put to use for solving complex problems.

2.1 Geospatial

New satellite systems like Sentinel-1/2, Planet Labs’ Dove Program, and countless other projects aim to provide nearly real-time imaging of the entire planet.

  • First we select our current set of images (LakeUrmia) from the archive
SELECT image FROM TimelapseArchive WHERE Location IS "Lake Urmia" 
  AND Date.Year BETWEEN 1984 AND 2012
Raw Images
  • Next we focus on the differences between the images
SELECT LA.image-LAB.image FROM LakeUrmia LA 
  JOIN LakeUrmia LAB ON DAYDIFF(LA.Date,LAB.Date)==1
Raw Images \(\rightarrow\) Final Forms

2.2 Selfie Tracking

Similar analyses can be performed on other time-lapse images like this selfie stream provided by Andy as he hiked the Pacific Crest Trail.

Raw Images

Next we focus on the differences between the images

SELECT SG.image-SGB.image FROM SelfieGallery SG 
  JOIN SelfieGallery SGB LAB ON DAYDIFF(SG.Date,SGB.Date)==1
Raw Images \(\rightarrow\) Raw Images

2.3 Model-Driven Approach

Rather than focusing only on the differences, we can utilize a model-based approach to analyze the data

\[\rightarrow \textrm{Segment different regions of the image based on land-cover}\]

SELECT Segment_Water(Image) as WaterArea FROM TimelapseArchive
Segment

\[ \downarrow \textrm{ Track the size and position of the lake} \]

SELECT CenterOfMass(roi) FROM WaterImage GROUP BY time

3 Learn More

4Quant is active in a number of different areas from medicine to remote sensing. Our image processing framework (Spark Image Layer) and our query engine (Image Query and Analysis Engine) are widely adaptable to a number of different specific applications.

3.2 Technical Presentations

To find out more about the technical aspects of our solution, check out our presentation at:

4 Acknowledgements

Data was obtained from the video. Analysis powered by Spark Image Layer from 4Quant, Visualizations and Document Generation provided by:

To cite ggplot2 in publications, please use:

H. Wickham. ggplot2: elegant graphics for data analysis. Springer New York, 2009.

A BibTeX entry for LaTeX users is

@Book{, author = {Hadley Wickham}, title = {ggplot2: elegant graphics for data analysis}, publisher = {Springer New York}, year = {2009}, isbn = {978-0-387-98140-6}, url = {http://had.co.nz/ggplot2/book}, }

To cite package ‘leaflet’ in publications use:

Joe Cheng and Yihui Xie (2014). leaflet: Create Interactive Web Maps with the JavaScript LeafLet Library. R package version 0.0.11. https://github.com/rstudio/leaflet

A BibTeX entry for LaTeX users is

@Manual{, title = {leaflet: Create Interactive Web Maps with the JavaScript LeafLet Library}, author = {Joe Cheng and Yihui Xie}, year = {2014}, note = {R package version 0.0.11}, url = {https://github.com/rstudio/leaflet}, }

To cite plyr in publications use:

Hadley Wickham (2011). The Split-Apply-Combine Strategy for Data Analysis. Journal of Statistical Software, 40(1), 1-29. URL http://www.jstatsoft.org/v40/i01/.

A BibTeX entry for LaTeX users is

@Article{, title = {The Split-Apply-Combine Strategy for Data Analysis}, author = {Hadley Wickham}, journal = {Journal of Statistical Software}, year = {2011}, volume = {40}, number = {1}, pages = {1–29}, url = {http://www.jstatsoft.org/v40/i01/}, }

To cite the ‘knitr’ package in publications use:

Yihui Xie (2015). knitr: A General-Purpose Package for Dynamic Report Generation in R. R package version 1.10.

Yihui Xie (2013) Dynamic Documents with R and knitr. Chapman and Hall/CRC. ISBN 978-1482203530

Yihui Xie (2014) knitr: A Comprehensive Tool for Reproducible Research in R. In Victoria Stodden, Friedrich Leisch and Roger D. Peng, editors, Implementing Reproducible Computational Research. Chapman and Hall/CRC. ISBN 978-1466561595

To cite package ‘rmarkdown’ in publications use:

JJ Allaire, Joe Cheng, Yihui Xie, Jonathan McPherson, Winston Chang, Jeff Allen, Hadley Wickham and Rob Hyndman (2015). rmarkdown: Dynamic Documents for R. R package version 0.7. http://CRAN.R-project.org/package=rmarkdown

A BibTeX entry for LaTeX users is

@Manual{, title = {rmarkdown: Dynamic Documents for R}, author = {JJ Allaire and Joe Cheng and Yihui Xie and Jonathan McPherson and Winston Chang and Jeff Allen and Hadley Wickham and Rob Hyndman}, year = {2015}, note = {R package version 0.7}, url = {http://CRAN.R-project.org/package=rmarkdown}, }

To cite package ‘DiagrammeR’ in publications use:

Knut Sveidqvist, Mike Bostock, Chris Pettitt, Mike Daines, Andrei Kashcha and Richard Iannone (2015). DiagrammeR: Create Graph Diagrams and Flowcharts Using R. R package version 0.7.

A BibTeX entry for LaTeX users is

@Manual{, title = {DiagrammeR: Create Graph Diagrams and Flowcharts Using R}, author = {Knut Sveidqvist and Mike Bostock and Chris Pettitt and Mike Daines and Andrei Kashcha and Richard Iannone}, year = {2015}, note = {R package version 0.7}, }